Papers with sequence generation task
Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised neural machine translation (UNMT) use cross-lingual pretraining to align the lexical- and high-level representations of two languages. |
| Approach: | They propose to use type-level cross-lingual subword embeddings to enhance the bilingual masked language model pretraining with lexical-level information to align the two languages. |
| Outcome: | Empirical results show that the method improves on UNMT (up to 4.5 BLEU) and bilingual lexicon induction compared to baseline models. |
A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis (2022.findings-naacl)
Copied to clipboard
| Challenge: | Pre-trained language models are often used to achieve state-of-the-art results . eval paper shows that generative language model can handle joint and multi-task settings . |
| Approach: | They propose to reformulate extraction and prediction tasks into a sequence generation task . they propose a generative language model with unidirectional attention that learns to accomplish the tasks via language generation . |
| Outcome: | The proposed model outperforms the state-of-the-art in few-shot and full-shot settings. |
Question Generation for Adaptive Education (2021.acl-short)
Copied to clipboard
| Challenge: | Existing systems depend on a pool of hand-made questions, limiting how fine-grained and open-ended they can be in adapting to individual students. |
| Approach: | They propose to fine-tune pre-trained language models for deep knowledge tracing to generate reversetranslation questions conditioned on the student and target difficulty. |
| Outcome: | The proposed model can generate well-calibrated language translation questions for second language learners from a real online education platform. |
Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement (D18-1)
Copied to clipboard
| Challenge: | Despite its success, neural autoregressive modeling has its weakness in decoding, i.e., finding the most likely sequence. |
| Approach: | They propose a conditional non-autoregressive neural sequence model based on iterative refinement based upon latent variable models and conditional denoising autoencoders. |
| Outcome: | The proposed model significantly speeds up decoding while maintaining the generation quality comparable to the autoregressive counterpart. |
UniverSLU: Universal Spoken Language Understanding for Diverse Tasks with Natural Language Instructions (2024.naacl-long)
Copied to clipboard
Siddhant Arora, Hayato Futami, Jee-weon Jung, Yifan Peng, Roshan Sharma, Yosuke Kashiwagi, Emiru Tsunoo, Karen Livescu, Shinji Watanabe
| Challenge: | Recent studies leverage large language models with multi-tasking capabilities, using natural language prompts to guide the model’s behavior and surpassing performance of task-specific models. |
| Approach: | They adapt a pre-trained automatic speech recognition model to additional tasks using single-token task specifiers. |
| Outcome: | The proposed model can generalize to new datasets and languages for seen task types. |
Leveraging WordNet Paths for Neural Hypernym Prediction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on lexical relations based on distributed representations has differed widely. |
| Approach: | They propose a model that generates taxonomy paths for hypernym prediction using WordNet sequences. |
| Outcome: | The hypo2path model outperforms the best model by 4.11 points in hit-at-one (H@1) The proposed model outpersforms previous models by a factor of 0.9. |
Cost-effective End-to-end Information Extraction for Semi-structured Document Images (2021.emnlp-main)
Copied to clipboard
| Challenge: | a real-world information extraction system for semi-structured document images often involves a long pipeline of multiple modules, which can lead to unstable performance if not designed carefully. |
| Approach: | They propose to use a sequence generation task to build an end-to-end IE system . they propose to combine three manually engineered modules with one data-driven module . |
| Outcome: | The proposed system can be easily replaced and deployed in large-scale production. |
A Multi-label Multi-hop Relation Detection Model based on Relation-aware Sequence Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods treat multi-label learning problem as a single label . Existing approaches focus on measuring semantic similarity of questions and candidate relations . |
| Approach: | They propose to solve multi-hop relation detection problem by generating sequences of hops and labels. |
| Outcome: | The proposed method is effective in KBQA, despite the unknown number of labels and hops. |
End-to-end Dense Video Captioning as Sequence Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for dense video captioning use a two-stage generative process . but, more complex tasks are not able to fully utilize this powerful paradigm . |
| Approach: | They propose to model two subtasks of dense video captioning as one sequence generation task and predict the events and the corresponding descriptions. |
| Outcome: | Experiments on YouCook2 and ViTT show that the proposed model can be used on any video platform. |
ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models with table-specific architectures and pre-training methods perform well on understanding table structures but lack table reasoning skills. |
| Approach: | They propose to pre-train tables with table reasoning skills without complex architectures . they define 7 table reasoning skill, and then pre-teach them to generate tables . |
| Outcome: | The proposed model improves on four tasks and is available on github. |
Discourse Structure-Aware Prefix for Generation-Based End-to-End Argumentation Mining (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in AM models overlook the integration of supplementary discourse structure information, resulting in suboptimal outcomes. |
| Approach: | They propose a framework which generates discourse structure-aware prefixes for each layer of the generation model. |
| Outcome: | The proposed framework achieves state-of-the-art performance on two AM benchmarks. |